Hierarchical Temporal Memory (HTM) networks represent a new approach to machine intelligence. In a HTM network, training data comprising temporal sequences of spatial patterns are presented to a network of nodes. The HTM network then builds a model of the statistical structure inherent to the patterns and sequences in the training data, and thereby learns the underlying ‘causes’ of the temporal sequences of patterns and sequences in the training data. The hierarchical structures of the HTM network allow them to build models of very high dimensional input spaces using reasonable amounts of memory and processing capacity.
FIG. 1 is a diagram illustrating a hierarchical nature of the HTM network where the HTM network 10 has three levels L1, L2, L3, with level L1 being the lowest level, level L3 being the highest level, and level L2 being an intermediate level between levels L1 and L3. Level L1 has nodes 11A, 11B, 11C and 11D; level L2 has nodes 12A and 12B; and level L3 has node 13. In the example of FIG. 1, the nodes 11A, 11B, 11C, 11D, 12A, 12B, and 13 are hierarchically connected in a tree-like structure such that each node has several children nodes (that is, nodes connected at a lower level) and one parent node (that is, node connected at a higher level). Each node 11A, 11B, 11C, 11D, 12A, 12B, and 13 may have or be associated with a capacity to store and process information. For example, each node 11A, 11B, 11C, 11D, 12A, 12B, and 13 may store input data (for example, sequences of spatial patterns) associated with particular object or a state of an object. Further, each node 11A, 11B, 11C, 11D, 12A, 12B, and 13 may be arranged to (i) propagate information up the HTM hierarchy to any connected parent node and/or (ii) propagate information down the HTM hierarchy to any connected children nodes.
The HTM training process is a form of unsupervised machine learning. However, in a training stage, indexes attached to the input patterns may be presented to the HTM network to allow the HTM to associate particular categories with the underlying generative causes. Once an HTM network has built a model of a particular input space, it can operate in an inference stage. In the inference stage, novel sequences of input patterns are presented to the HTM network, and the HTM network will generate a ‘belief vector’ that provides a quantitative measure of the degree of belief or likelihood that the input pattern was generated by the underlying cause associated with each of the indexed categories to which the HTM network was exposed in the training stage.